Measuring Hint Level in Open Cloze Questions

Providing the first few letters of a missing word in a sentence gives information about this word. This paper attempts to measure the information transmitted in that case. In order to do so, we analyzed response accuracy for open cloze questions, that is fill-in-the-blank questions without multiple choice answers. In this study, native and non-native speakers of English answered a series of open cloze questions that were semi-automatically generated. Hints were provided that consisted of the first few letters of the missing word. Results showed that question difficulty, hence the quantity of information transmitted, is related to the number of letters that are provided, to physical properties of these letters and to syllables formed by these letters. Performances did not appear to depend on letter or syllable frequency. Controlling hint level in a word completion task is critical in order to provide practice exercises adapted to student levels.